Clone Detector Evaluation Can Be Improved: Ideas from Information Retrieval

We believe the techniques for evaluating clone detectors can be improved, and that the improvements can lead the way to better clone detector research and research results. Current techniques are based on simple performance measures borrowed from information retrieval (IR) research. Here we argue that additional IR evaluation measures can be usefully imported. 1. Limitations of current research Automated methods for finding software clones have, in the past, been primarily evaluated according to simple measures adapted from prior work in the field of information retrieval (IR). The main stated purpose of an IR system is to enable users to find documents which are relevant to the user based on some approximation of their needs in the form of a query. One classic way of evaluating IR systems is to examine their scores on precision and recall. These measures are usually neatly defined and related by way of a table (see e.g., van Rijsbergen [5]):